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REMARKS — General 

By the above amendment, Applicants have amended the claims to define the invention more 
particularly and distinctly so as to overcome the technical rejections and define the invention 
patentably over the prior art. 
The Rejection Of The Claims Under § 112 

The last O.A. rejected claims 1-16 under 35 U.S.C. 1 12, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicants 
regard as the invention. 

Applicants canceled the claims 1-16 and wrote new claims 17-36. The limitations in the claims 
1-16 are incorporated into the newly written claims 17-36 to more particularly define the 
invention in a patentable manner, according to the O.A. 
Accordingly applicants request reconsideration of this rejection. 

Regarding claim 1, the last O.A. noted that the scope of the claim is unclear as to whether the 
claim is attempting to claim a method or an apparatus, and the claim should recite one or the 
other. 

According to the last O. A., the applicants separated the claim 1 and wrote new claims 1 7 and 27, 
which incorporate the limitations in the claim 1 . Claim 1 7 claims a method that incorporates all 

« 

the method subject matter of claim 1 and adds additional subject matter. Claim 27 claims an 
apparatus that incorporates all the apparatus subject matter of claim 1 and adds additional subject 
matter. 

The last O.A. also noted that claim 8 recites the limitation "the color model" in Ime 1, but there 
is insufficient antecedent basis for this limitation in the claim. 
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Applicants rewrote the claim 8 as a new claim 21 to have proper antecedent basis for the 
limitation in the claim and to more particularly define the invention in a patentable manner, 
according to the O.A. 

Accordingly applicants request reconsideration of the rejection, as now applicable to claim 21. 
The Rejection Of The Claims Under § 102 

The O.A. rejected claims 1, 3, 5-7, and 9-14 under 35 U.S.C. 102(e) as being unpatentable by 

U.S. Patent No. 6985620 (Sawhney, et al). 

Applicants amended the claims as follows: 

The Rejection Of Claim 1 on Sawhney, et al. Overcome 

Regarding claim 1, the O.A. noted that Sawhney, et al. discloses a robust apparatus and method 
for visually tracking an object in three dimensions (fig. 4, and the apparatus that carries out fig. 
4). 

Claim 1 has been canceled and all of the limitations in claim 1 have been incorporated into the 
newly written claim 1 7 as limitations of the newly made independent claim to define patentably 
over the reference. 

Applicants request reconsideration of the rejection, as now applicable to claim 17, for the 
following reasons: 

(1) Novel and unobvious approaches in applicants' system are clearly foreign to Sawhney, et al., 
and they were not suggested in the reference or any prior art. Several differences in the 
fundamental approaches are discussed later. The significant differences in the approaches 
between applicants' present invention and Sawhney, et al. are not to be underestimated or 
misunderstood. 
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(2) The novel steps, such as the usage of a large number of target location hypotheses and the 
combination of the spread out color and motion cue images, in the present invention were not 
suggested in the reference or any prior art, produce new and unexpected results, and hence are 
imobvious and patentable over the reference. 

(3) The reference does not explicitly disclose the solution for the problems in the task of 3D 
tracking of object, e.g. impracticality of model based tracking, which the present invention 
overcomes. The reference takes a different approach from that of the present invention for 
solving the task. Thus, there is no logical explanation that the novel and unobvious features in 
the present invention can be derived from the reference. 

The Reference and Differences of the Present Invention Thereof 

In regards to the above three points, applicants will discuss the reference and the novelty of the 
present invention and its unobviousness over the reference hereinafter. 

Sawhney, et al. disclosed a method for estimating a pose of a camera using a three-dimensional 
model of a scene. Overall, Sawhney, et al. does not show the novel features in applicants' present 
invention, and the differences can comprise the following: 

(1) Target object tracking based on novel usage of multiple target location hypotheses is clearly 
foreign to Sawhney, et al. 

(2) Combination of spread out color and motion cues is clearly foreign to Sawhney, et al. 

(3) The applicants' solution in the maimer proposed by the present invention to overcome the 
limitation of the model based approach in the specific application area is foreign to Sawhney, et 
al. 
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(4) Random distribution of the hypotheses is foreign to Sawhney, et aL, and the Focus of the idea 
in Sawhney, et al. is contradictory to that of the present invention. 

(5) Handling arbitrary shape and contour of the target object is foreign to Sawhney, et al. 
Target Object Tracking Based On Novel Usage Of Multiple Target Location Hypotheses Is 
Clearly Foreign To Sawhney, et aL 

Applicants noted, "Projective geometry states, that we can obtain from the projected locations ri 
and rR of a point R, the corresponding 3D coordinate through triangulation. Most stereo 
algorithms rely on this principle by finding correspondences between two images and recovering 
depth through triangulation. However, finding correspondences between views is challenging," 
(Applicants, page 10, lines 3-7). 

As a solution to the challenge, applicants disclosed a novel approach to the problem with the 
measurement of the confidence in the hypothesis as in "given a hypothesized hand location in 
world coordinates R, the image data is used to measure the confidence in this hypothesis, i.e., the 
degree to which the image data supports the hypothesis that a hand is indeed present at the 2D 
locations corresponding to the world coordinate R" (Applicants, page 10, lines 11-14). 
Especially, to increase the accuracy of the approach, applicants disclosed, "a large number of 
such hypotheses are maintained and combined to get a statistical measure of the true hand 
location in world coordinate space" (Applicants, page 10, lines 14-16). "In the illustration FIG. 
1 1, the value is N=14 but in practice, this number is much larger." (Applicants, page 16, lines 7- 
8). Typically, in an exemplary embodiment of the present invention, the number of hypotheses 
could be between 40 and 500, but depending on the parameters, such as the size of a hand, the 
number could be more or less. 
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The usage of "a large number of such hypotheses" is clearly foreign to Sawhney, et al. The rigid 
model based approach in Sawhney, et al., as shown in "The method begins with a three 
dimensional model of the scene, 400, such as that shown in FIG. 2. The first step, 402, is to 
generate an initial, rough estimate of the pose of an image by comparing the image to the three 
dimension model of the scene." (Sawhney, et al., column 4, lines 55-59), and the usage of 
relevant features from the model, such as 3D line segments, in Sawhney, et al, as shown in "a 
set of relevant features is selected from the three dimensional model based on the estimate of the 
pose, step 404." (Sawhney, et al., column 5, lines 21-23 and lines 32-33), are clearly different 
approaches from applicants' novel usage of multiple hypotheses for target object tracking. 
Combination Of Spread Out Color And Motion Cues Is Clearly Foreign To Sawhney, et al. 
Applicants disclosed a "spread out" post-processing of the color and motion confidence images, 
"the color and the motion cues are spatially spread out using averaging before they are combined. 
Given an image 800 with each location in the image containing the value of a cue response, a 
new image 810 is generated, where a location contains the thresholded average of values in a 
neighborhood surrounding the location." (Applicants, page 13, lines 14-17). This novel approach 
is to provide a solution to a problem of biased responses in the confidence images, as in "While 
the motion cue function M(s,t) (630) for this target tends to have high responses at the edges 63 1 
and 632 of the target, the color cue function 620 tends to have high responses at the interior of 
the target 621" (Apphcants, page 13, lines 8-11). 

The "spread out" post-processing is clearly foreign to Sawhney, et al., and furthermore the novel 
usage of a combination of the "spread out" post-processed color and motion confidence images 
produces new and unexpected results and is clearly foreign to Sawhney, et al. 
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The Applicants' Solution In The Manner Proposed By The Present Invention To Overcome 
The Limitation Of The Model Based Approach In The Specific Application Area Is Foreign 
To Sawhney, et al. 

It is well known that the fixed size and rigid model shape in Sawhney, et al. cannot handle the 
tracking of the complex target object, such as a human hand, in a 3D physical space in real-time, 
in the manner proposed by applicants. The size and shape of a human hand in the captured 
images can change dynamically and arbitrarily in a real world application of 3D hand tracking. 
Therefore, it is one of the objectives of the present invention to overcome such impracticality by 
the suggested novel approaches in the present invention. Applicants' system utilizes a large 
number of hypotheses about the actual 3D object location based on color and motion confidence 
cue images rather than the model of the object (Applicants, page 5, line 16 - page 6, line 3). 
As a matter of fact, the specification of the present invention already pointed out the problem, in 
the discussion of the background of the invention, "It is well known in the art, that this feature 
matching approach takes a lot of computational resources and that it easily fails in situations 
where no or very few clean feature sets can be extracted, where occlusion prevents pairing of a 
feature in one image with a point in the second image. It is common to pair two features that do 
not correspond to the same location in space, yielding an entirely incorrect 3D location 
estimate.'' in regards to the International Patent, WO 02/07839 A2, Hildreth, et al. (Applicants, 
page 3, lines 4-9) and "however, model based tracking as described in their paper is 
computationally extremely expensive and not suitable for practical application." in regards to 
Gonclaves, et al. and Filova, et al. (Applicants, page 4, lines 21-22). 
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As Sawhney, et al. suggested, the 3D line segments-based relevant feature selection in their 
approach could face problems of clutter, occlusion, and multiple local minima (Sawhney, et al., 
colunm 5, lines 41-67). In some cases, the target object may not be seen entirely within the field 
of view of the cameras, thus the pose of the model cannot be correctly calculated (Sawhney, et 
al., column 8, lines 26-35). Applicants' present invention provides a novel approach to overcome 
the problems that the 3D line segments-based relevant feature selection in Sawhney, et al. could 
face because the present invention does not rely on the linear line segments that are prone to the 
noises and obstacles. 

Furthermore, as Sawhney, et al. said in "One possible use of the present invention is illustrated in 
FIGS. 8 and 9. This application of the present invention involves using current video from a 
camera mounted on a remote vehicle to locate the vehicle relative to objects in a three 
dimensional model. An additional application for the present invention may be to extend and 
refine a three dimensional model of an area as the area is explored by the remote vehicle possibly 
beyond the range of the scene originally covered by the model. Numerous applications in 
robotics exist for such methods. These methods may be particularly useful for the exploration of 
places in which other location techniques, such as GPS, are not practical, including underground, 
deep-sea, and interplanetary exploration." (Sawhney, et al., column 10, lines 50-63), the mainly 
focused application area of the Sawhney, et al. is totally different from that of the present 
invention, which is mainly tracking of human hands in front of a interactive system. The 
fundamental technologies and problems to overcome are totally different between the present 
invention and Sawhney, et al. 
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Therefore, the novel approaches in solving the problem for the specific application area of the 
real time hand tracking in the manner proposed by the present invention, which overcomes the 
model based approach, are clearly foreign to Sawhney, et al. 

Random Distribution Of The Hypotheses Is Foreign To Sawhney, et ah And The Focus Of 
The Idea In Sawhney, et al. Is Contradictory To That Of The Present Invention 

Applicants disclosed random distribution of the hypotheses at the starting point of the hand 

tracking, "The target location hypotheses are generated over time as follows: When the 
method starts (say at time t=0), and no information jfrom the past is available, the R[ at time t=0 

are randomly distributed in the interaction volume 400. For all later times t>0, N new hypotheses 
are created from the N old hypotheses available from the previous time step t-dt as follows: For 
the creation of the i-th {i between 1 and N) hypothesis, a number j between 1 and N is randomly 

drawn such that the probability of drawing a number n is equal to the probability 

(Applicants, page 17, lines 2-8). This approach is clearly foreign to Sawhney, et al 
Furthermore, at the initial estimate step, Sawhney, et al. suggests the usage of a user's manual 
input for setting the pose for the first frame as one of the initial estimate methods (Sawhney, et 
al., column 4, line 60), which the present invention identifies as a problem in an automatic object 
tracking system and the novel approach of the present invention overcomes. Applicants' 
preliminary preparation step, such as the calibration of the cameras, uses manual input for 
determining the image locations of the calibration point, but the tracking of the target object in 
applicants' system is processed completely automatic without requiring any manual initialization. 
This further shows the approach and focus of the idea in Sawhney, et al. is contradictory to that 
of the present invention. 
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Handling Arbitrary Shape And Contour Of The Target Object Is Foreign To Sawhney, et 
al. 

As discussed above, applicants' system is based on a novel usage of multiple hypotheses for 
target object tracking, and "In step 1440, N new hypotheses are created from the hypotheses at 
the previous time step t-dt by randomly selecting previous hypotheses with probability equal to 
the hypothesis weights" (Applicants, page 19, lines 11-13). Therefore, applicants' present 
invention is not bound by any fixed shape or contour of a certain target object model. This 
capability of handling any arbitrary shape and contour of the target object is foreign to Sawhney, 
et al. 

As an example of the application-dependent heuristics for the culling process, Sawhney, et al. 
suggests "keeping only model edges common to exactly two faces, keeping edges only at large 
dihedral angles, ignoring edges which are too close together, and ignoring edges near and on the 
ground" (Sawhney, et al., column 6, lines 2-5). However, the location of the hand cannot always 
rely on model edges conunon to exactly two faces, unlike the simple and rather linearly edged 
polygons that Sawhney, et al. used. 

Sawhney, et al. also argues that "While the pose, and hence occlusions, may change during the 
optimization process, it may suffice to cull model edges once when initializing the optimization 
and afterwards only if the pose changes significantly." (Sawhney, et al., colunm 6, lines 12-15). 
However, considering the fact that human hand shape can arbitrarily change, there is no 
justification that this assumption can support the continuous automatic tracking of a human hand. 
Therefore, the novel steps, such as the usage of a large number of target location hypotheses and 
the combination of the spread out color and motion cue images, in the present invention were not 
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suggested in the reference or any prior art, produce new and unexpected results, and hence are 
unobvious and patentable over the reference. 

Furthermore, the reference does not explicitly disclose the solution for the problems in the task 
of 3D tracking of object, e.g. impracticality of model based tracking, which the present invention 
overcomes as set forth above. The reference takes a different approach from that of the present 
invention for solving the task. Thus, there is no logical explanation that the novel and unobvious 
features in the present invention can be derived from the reference. 
The Rejection Of Claim 3, 5-7, and 9-14 on Sawhney, et aL 
Applicants amended the claims 3, 5-7, and 9-14 as follows: 
Claim 3 has been canceled according to the last O.A. 

Claims 5, 6, and 7 have been canceled, and the limitations in claims 5, 6, and 7 have been 
incorporated into the newly v^itten claims 18, 20, and 21, respectively, to more particularly 
define the invention in a patentable maimer over the cited prior art. 

Claims 9-13 have been canceled, and the limitations in claims 9-13 have been incorporated into 
the newly written claims 22-26, respectively, to more particularly define the invention in a 
patentable manner over the cited prior art. 

Claim 14 has been canceled, and the limitations in claim 14 have been incorporated into the 
newly written claim 1 7, to more particularly define the invention in a patentable maimer over the 
cited prior art. 

The Dependent Claims Are a Fortiori Patentable Over Sawhney, et aL 

New dependent claims 18 to 26 incorporate all the subject matter of claim 17 and add additional 
subject matter, which makes them a fortiori and independently patentable over the reference. 
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Claim 18 further adds a step of projecting target locations with projections that are obtained by 
calibrating said imaging sensors with respect to a reference coordinate system. Applicants 
disclosed the usage of an exemplary calibration cube with eight calibration points, "as illustrated 
in Fig. 9, the camera calibration matrices are obtained by placing a calibration target such as a 
cube (1000) with a set of Nc calibration points (e.g., 1010a, 1010b) with known world 
coordinates R^^ in the viewing area. In FIG. 9, the calibration cube has Nc=S calibration points." 
(Applicants, page 17, line 16 - page 18, line 17). Sawhney, et al. discloses the projection of the 
relevant model features into the estimated pose (Sawhney, et al., fig. 4, item 406). The step of 
projecting a large number of target location hypotheses, not relevant model features, based on the 
calibration process in the manner proposed by the present invention is not shown in Sawhney, et 
al. 

In addition, in Sawhney, et al. "projected model features are culled out if any similarly oriented 
projected model edge lies near it. Projected model features may also be culled if image texture 
analysis reveals the presence of multiple image edges within a predetermined distance." 
(Sawhney, et al., column 5, lines 60-64), whereas in applicants' system the confidence image is 
calculated based on all the projected hypotheses, " CONF(s,t) is an image at time t that contains 
for each location s=(x,y) a measure of the confidence of a hand being present at the world 
location that projected to the image coordinate (Applicants, page 14, lines 17-19). In a later 
process, applicants' system utilizes the projected hypotheses to see whether the location has 
aggregated confidences from the multiple images sources, and if a projected hypothesis does not 
support the confidence image, such as in the case of "940C" for the "location 940" (Applicants, 
fig. 8, item 940 and item 940C), the "location 940 not on the users hand, that projects to 
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locations 940b in the left confidence image and to location 940c in the right image only shows a 
high confidence in one of the two cameras." (Applicants, page 15, lines 5-7). 
Therefore, not only the target objects that are projected are different, but also the idea, processes, 
and usage of the projection between Sawhney, et al. and applicants' system are clearly different. 
Claim 19 fiirther adds a step of measuring overall confidence of the target object location based 
on a combination of a plurality of confidence images from each of the imaging sensors. Sawhney, 
et al. is clearly foreign to this. 

Claim 20 further adds a step of measuring the confidences based on a combination of color and 
motion cues in each of the images, wherein the color and motion cues are spatially spread out 
using averaging before they are combined. As stated above, this feature is novel with applicants 
and produces new and unexpected results — the combination of the spread out images results in 
an image where a combined response is not only present at some of the edges of the target object 
but also contains strong responses in the interior of the target object. This is clearly foreign to 
Sawhney, et al. 

Claim 21 fiirther adds a step of calculating the color cues using a color model of the target object, 
wherein the color model of the target object is represented by a histogram that is estimated by 
collecting color samples of the target object. Sawhney, et al. mentioned "color" (Sawhney, et al., 
col. 5, lines 30-31) as one of the other ways to define other types of edges (Sawhney, et al., col. 5, 
lines 27-31). The novel usage of color for calculating the color cues of the confidence measure, 
not the edges of models, is clearly foreign to Sawhney, et al. 

Claim 22 further adds a step of calculating motion cues by measuring differences between 
images captured sequentially by the imaging sensors. Sawhney, et al. discloses, "assumed 
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motion" from the previous frame's estimated pose can be used for predicting "initial pose" 
(Sawhney, et al. col. 4, lines 64-67). Here, Sawhney, et al. used the "assumed motion" only for 
predicting the "initial pose", whereas applicants' system actually measures the motion, not 
assuming, and calculates the motion cues throughout the entire tracking of the target object, not 
just for the "initial pose" based on the novel approach in the motion cues calculation (Applicants, 
page 1 1, line 1 - page 12, line 2). 

Sawhney, et al. also discloses, optical flow can be used as "one method for determining an initial 
pose estimate" (Sawhney, et al. coL 4, line 64-col. 5, line 8). Again, Sawhney, et al. explicitly 
mentions that the "optical flow" can be used for the "initial pose" estimation, whereas 
applicants' system extracts color triplets, "from each of the two images, denoted as 
(Ys,t,Us,t,Vs,t) and (Ys,t-dt,Us,t-dt,Vs,t-dt).", and measures the differences for the motion cues 
throughout the entire tracking of the target object (AppUcants, page 12, lines 3-20). The novel 
usage of motion for calculating the motion cues of the confidence measure is clearly foreign to 
Sawhney, et al. 

Claim 23 fiirther adds a step of maintaining 3D target location hypothesis by creating a set of 3D 
target location hypotheses at each time step. The target location hypothesis maintenance 
disclosed in the manner proposed by the applicants' system (Applicants, page 17, lines 1-14) is 
clearly foreign to Sawhney, et al., which uses selected "3D line segments from the model as 
relevant features" (Sawhney, et al., column 5, lines 21-34). As discussed above, the novel usage 
of a large number of hypotheses in the applicants' system is to overcome the obstacles that the 
"3D Hne segments from the model as relevant features" in the model based approach of Sawhney, 
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et al. has. Furthermore, the idea of calculating the weights of the new hypotheses by the 
confidences is further foreign to Sawhney, et al. 

In addition, Sawhney, et al. discloses, "The first step, 402, is to generate an initial, rough 
estimate of the pose of an image by comparing the image to the three dimension model of the 
scene. Numerous methods may be used to obtain this initial estimate. In an interactive system, a 
user can set the pose for the first frame. Physical measurements from position/attitude sensors 
mounted on the camera platform may be another source. Alternatively, when processing a 
sequence of fi-ames, the estimated pose of the previous fi-ame may be used as the initial pose, or 
the pose can be predicted firom the previous frame's estimated pose based on assumed motion." 
(Sawhney, et al., column 4, lines 57-67), in which the "first step; 402" (Sawhney, et al., fig. 4, 
item 402) is out of the loop that exist among the following steps in the flowchart, as explicitly 
depicted in the fig. 4. Therefore, the novel step of creating a set of 3D target location hypotheses 
based on the confidence fimction CONFlr(R. t) at each time step is clearly foreign to Sawhney, et 
al. (Applicants, page 14, lines 21-22). 

Claim 24 further adds a step of creating said 3D target location hypotheses based on known 3D 
target location hypotheses from a previous time step. Sawhney, et al. disclosed, "Alternatively, 
when processing a sequence of frames, the estimated pose of the previous frame may be used as 
the initial pose, or the pose can be predicted from the previous fi:ame's estimated pose based on 
assumed motion." (Sawhney, et al., column 4, lines 64-67). However, Sawhney, et al. does not 
create the 3D target location hypotheses, as discussed above, and furthermore Sawhney, et al. is 
foreign to the idea of creating the 3D target location hypotheses based on known 3D target 
location hypotheses from a previous time step. 
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The prediction of the initial pose from the previous frame's estimated pose in Sawhney, et al. is 
different from the creation of a large number of 3D target location hypotheses based on known 
3D target location hypotheses from a previous time step in applicants' system. As explicitly 
disclosed in "For all later times t>0, N new hypotheses are created from the N old hypotheses 
available from the previous time step t-dt as follows: For the creation of the i-th (i between 1 and 
N) hypothesis, a number j between 1 and N is randomly drawn such that the probability of 
drawing a number n is equal to the probability W"_^ . The location of the new i-th hypothesis is 

given by the location of the old hypothesis plus a small offset AR^ = (AAT/ , AF/ , AZ/ ) , where 

each of the three offsets AA"/ , AY/ , AZ/ are randomly drawn between two numbers -OFFSET 
and +OFFSET (- 5mm and +5mm for the preferred embodiment described here). The weight of 
the new hypotheses with location i?/ = + A/?/ is given by the confidence fimction 

CONFlrCRJ)-'' (Applicants, page 17, lines 5-14), the N hypotheses may contribute to the target 
location calculation where a number j between 1 and N is randomly drawn such that the 
probability of drawing a number n is equal to the probability W"_^ , but they are not identical to 

the estimated pose. This novel approach is clearly foreign to Sawhney, et al. 
Claim 25 fiirther adds a step of initially distributing the 3D target location hypotheses randomly 
in the space viewed by said imaging sensors. Sawhney, et al. discloses exemplary polyhedral 
model of scene in fig. 2 (Sawhney, et al, column 4, lines 32-33, fig. 2) and illustration of 
selection of relevant features 302 from the polyhedral model in fig. 3 (Sawhney, et al., column 5, 
lines 34-35, fig. 3). Sawhney, et al. clearly states, "All objects in this model are all represented as 
polyhedra composed of planar polygonal surfaces 200, even a curved object, such as cylinder 
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106. To accomplish such a polyhedral representation, the modeled representation of cylinder 106 
contains artificial edge boundaries 204. Also shown in FIG. 2 are dotted lines 202, which 
represent edge boundaries that are hidden in the illustrated view of the model." (Sawhney, et al., 
column 4, lines 33-40). The random distribution of the 3D target location hypotheses in 
applicants' system is not the same as the "polyhedral representation", in which even a curved 
object, such as cylinder, is represented as polyhedra composed of planar polygonal surface. This 
approach is clearly foreign to Sawhney, et al. 

Furthermore, as the last O.A. also noted, the exemplary polyhedral model of scene is reduced to 
only the relevant features in fig. 3 (Sawhney, et al., fig. 4, step 404), and Sawhney, et al. 
discloses details of the problems in selecting the relevant features and the culling process, 
whereas in applicants' system, "When the method is started (step 1400) at time t=0, N 
hypotheses are initially randomly distributed in the viewing volume 400 defined by the two 
cameras 1 10 and 120." (Applicants, page 18, line 21- page 19, line 1), and the N number of 
hypotheses are maintained through the target object tracking (Applicants, page 19, lines 11-13). 
This is completely foreign to Sawhney, et al. 

Claim 26 fiirther adds a step of adding random displacements to the location hypothesis at each 
time step for maintaining 3D target location hypotheses. In applicants' system, "each of the three 

offsets AZ/ , Al^' , AZ/ are randomly drawn between two numbers -OFFSET and +OFFSET (- 

5mm and +5mm for the preferred embodiment described here)" (Applicants, page 17, lines 1 1- 
13). This is clearly foreign to Sawhney, et al. 

Sawhney, et al. discloses, optical flow and RANSAC can be used for the initial pose estimation 
from the set of 3D and 2D correspondences (Sawhney, et al., column 5, lines 2-17). The 
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RANSAC algorithm, by M. A. Fischler, R. C. BoUes. Random Sample Consensus: A Paradigm 
for Model Fitting with Applications to Image Analysis and Automated Cartography. Comm. of 
the ACM, Vol 24, pp 381-395, 1981, iteratively selects a random subset of the original data 
points with the assumption that the probability of randomly selecting the inliers in the random 
subset is high. The RANSAC algorithm may help to find optimal edges, such as the "brightness 
edges" of the polygonal model in Sawhney, et al. However, Sawhney, et al. is clearly foreign to 
the idea of drawing random displacements and adding them to the target location hypotheses. 
Accordingly, applicants submit that the dependent claims are a fortiori patentable and should 
also be allowed. 

The Rejection Of The Claims Under § 103 

The O.A. rejected claims 2, 4, 8, 15, and 16 under 35 U.S.C. 103(a), and applicants amended the 
claims as follows: 

The Rejection Of Claims 2 and 4 on Sawhney, et al. and Ostenvell, et al. 

The O.A. rejected claims 2 and 4 under 35 U.S.C. 103(a) as being unpatentable over Sawhney, et 
al. in view of U.S. Patent No. 7106885 (Osterwell, et al.). 
Applicants amended the claims as follows: 

Claim 2 has been canceled according to the last O.A., and the color cameras in claim 2 are 
incorporated into the whereby clause of the newly written independent claim 27 as only 
exemplary imaging sensors. 

Claim 4 has been canceled according to the last O.A., and the IEEE 1394 cameras in claim 4 are 
incorporated into the whereby clause of the newly written independent claim 27 as only 
exemplary imaging sensors. 
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The Rejection Of Claim 8 on Sawhney, et al. and Brumitt Overcome 

The O.A. rejected claim 8 under 35 U.S.C. 103(a) as being unpatentable over Sawhney, et al. in 
view of U.S. Patent No. 6658136 (Brumitt). 
Applicants amended the claims as follows: 

Claun 8 has been canceled, and the limitations in claim 8 have been incorporated into the newly 
written claim 21 in the wherein clause to more particularly define the invention in a patentable 
manner over the cited prior art. New dependent claim 21 incorporates all the subject matter of 
claim 1 7 and adds additional subject matter, which makes it a fortiori and independently 
patentable over the reference. 

Regarding claim 21, it further adds a step of calculating the color cues using a color model of the 
target object, and the novel usage of color for calculating the color cues of the confidence 
measure, not the edges of models, is clearly foreign to Sawhney, et al. as set forth above. 
Furthermore, claim 2 1 adds a limitation in the wherein clause, which states the color model of 
the target object is represented by a histogram that is estimated by collecting color samples of the 
target object. 

As the last O.A. noted, Brumitt disclosed color histogram technique as an exemplary method to 
characterize the identified region and stored images (Brumitt, column 3, lines 9-21). 
Color histogram has been known in the practiced art and used in various applications, such as for 
the characterization of the identified region in Brumitt. However, the novel usage of the color 
histogram for calculating the color cues of the confidence measure for a large number of 
hypotheses in the maimer proposed by the applicants' system is foreign to Sawhney, et al. and 
Brumitt, or any combination thereof 
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The Rejection Of Claims 15 and 16 on Sawhney, et aL and Maki, et al. 

The O.A. rejected claims 15 and 16 under 35 U.S.C. 103(a) as being unpatentable over Sawhney, 
et al. in view of U.S. Patent No. 6072903 (Maki, et al.). 
Applicants amended the claims as follows: 

Claim 15 has been canceled according to the last O.A., and the human appendage in claim 15 is 
incorporated into the whereby clause of the newly written independent claim 1 7 as only 
exemplary target object. 

Claim 16 has been canceled according to the last O.A., and the human head in claim 16 is 
incorporated into the whereby clause of the newly written independent claim 17 as only 
exemplary target object. 
Newly Added Apparatus Claims 

Regarding claim 1, the last O.A. noted that the scope of the claim is unclear as to whether the 
claim is attempting to claim a method or an apparatus, and the claim should recite one or the 
other. According to the last O.A. regarding claim 1, the applicants separated the claim 1 and 
wrote new apparatus independent claim 27, which incorporates all the apparatus subject matter 
of claim 1 and adds additional subject matter, as set forth above. 

Applicants also wrote new apparatus dependent claims 28-36, which recite limitations that are 
similar and in the same scope of invention as to those in claims 18-26 above. 
Therefore, applicants request consideration of the newly written claims 28-36 for the same 
reasons as stated above in regards to claims 1 8-26, respectively. 
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CONCLUSION 

For all the above reasons, applicants subniit that the specification and claims are now in proper 
form, and that the claims all define patentably over the prior art. Therefore they submit that this 
application is in condition for allowance now, which action they respectfully solicit. 

Conditional Request for Constructive Assistance 

Applicants have amended the specification and claims of this application so that they are proper, 
definite, and define novel structure, which is also unobvious. If, for any reason this application is 
not believed to be in full condition for allowance, applicants very respectfully request the 
constructive assistance and suggestions of the Examiner pursuant to M.P.E.P. § 2173.02 and § 
707.07(j) in order that the undersigned can place this application in allowable condition. 

Very respectfiiUy, 





Nils Krahnstoever 



Emilio Schapira 




Rajeev Sharma 



Applicants Pro Se 



403 South Allen Street, Suite 101 



State College, PA 16801 



Tel. (814) 867-8977; Fax (814) 867-8957 
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Certificate of Mailing 

I hereby certify that this correspondence, and attachments, if any, will be deposited with the 
United States Postal Service by First Class Mail, postage prepaid, in an envelope addressed to 
"Box Non-Fee Amendments, Conmiissioner for Patents, P.O. Box 1450, Alexandria, VA 22313- 
1450" on the date below. 



